Skip to content

feat: support injection of CompressionCodecName, read DataSize#8

Open
skradel wants to merge 2 commits intostrategicblue:masterfrom
skradel:skradel-compression-codec
Open

feat: support injection of CompressionCodecName, read DataSize#8
skradel wants to merge 2 commits intostrategicblue:masterfrom
skradel:skradel-compression-codec

Conversation

@skradel
Copy link

@skradel skradel commented Dec 23, 2025

Zstd compression can yield superior compression as compared to Snappy on some types of input data.

  • Add a new ParquetWriter factory method to enable user selection other than the default.
  • Add method to read buffered+written data size of the underlying Writer to support occasional external flushing
  • (Somewhat crudely) Update the single unit/integration test to ensure both Snappy and Zstd function and that the codec's name appears in the first metadata block

@skradel skradel force-pushed the skradel-compression-codec branch 2 times, most recently from 5e7eea1 to f5811d4 Compare December 23, 2025 23:04
@skradel skradel force-pushed the skradel-compression-codec branch from f5811d4 to 77122a1 Compare December 23, 2025 23:05
@skradel skradel changed the title feat: support injection of CompressionCodecName feat: support injection of CompressionCodecName, read DataSize Dec 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant